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The primary goal of the Intensified Algebra I (IA) program is to 
enable mathematically underprepared students to successfully 
complete Algebra I in 9 th grade and stay on track to meet increasingly 
rigorous high school mathematics graduation requirements. The 
program was designed to bring a range of both cognitive and non- 
cognitive supports to bear on underprepared students’ learning of 
rigorous algebra content within an extended period of instruction, 
thus allowing them to catch up with their peers on a pathway toward 
more advanced mathematics courses. This study measured gains in 
IA students’ overall mathematical performance based on a 
comprehensive multiple-choice assessment of algebra proficiency and 
a constructed response assessment. Results showed IA students’ 
performance significantly increased over the academic year on both 
assessments. In addition, students’ performance showed a 
consistently large improvement in three of six core content areas 
(Graphing Linear Equations, Functions and Graphs, and Solving 
Linear Equations) within the multiple-choice assessment. This study 
provides promising evidence of IA meeting its programmatic goal of 
supporting underprepared students learning of core algebra content 
in a function-based curriculum. Implications for curriculum 
implementation and ongoing development, and further research are 
discussed. 
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The importance of students' success in Algebra I for 
college and career options has become a focal point of 
mathematics education research and practice over the last two 
decades, leading to the familiar characterization of introductory 
algebra as a gatekeeper course (Moses, 2001). The connection 
between success and failure in introductory algebra and other 
key academic outcomes is well documented. Students who are 
successful in introductory algebra are more likely to take 
advanced mathematics courses, graduate from high school, and 
achieve academic success beyond high school (Adelman, 2006; 
Ma & Wilkins, 2007). Conversely, students who fail algebra 
run a greater risk of dropping out of high school and losing 
future opportunities for academic and economic success, as 
well as full participation in democratic citizenry (Moses, 2001). 
It is in this context that the Intensified Algebra I (IA) program 
was developed with the aim of providing underprepared 
students the opportunity to learn in a meaningful way the core 
ideas of algebra, with adequate supports to be successful (Agile 
Mind, The Charles A. Dana Center, & University of Illinois at 
Chicago Learning Sciences Research Institute, 2013). 

Researchers have documented extensively the difficulties 
students have encountered in learning algebra (e.g., Boulton- 
Lewis, Cooper, Atweh, Pillay, & Wills, 2001; Demana & 
Leitzel, 1988; MacGregor, 1996). In parallel, the structural 
approach to algebra used in most traditional programs has been 
identified as problematic (Cai, Nie, & Moyer, 2010). Such an 
approach introduces equation solving not only before variables 
and functions but it is also characterized by abstract work with 
symbols not grounded on conceptual foundations developed in 
problem situations. In response, several alternative approaches 
to the teaching of algebra have been developed in the last 
decades (e.g., Mason, 1996; Schwartz & Yerushalmy, 1992). 
One such approach is functions-based, which is one of the 
central aspects of the IA design. In this approach, functional 
relationships are introduced in situations and contexts before 
equation solving, and contextualized functions are used as a 
way to provide meaning to equations. 

This paper focuses on IA students’ performance to better 
understand the extent to which IA supports students’ 
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acquisition of key mathematics knowledge and skills in the 
domain of algebra. Specially, the research questions guiding 
the report are: 

1. Do Intensified Algebra students show significant 
performance gains in mathematics content typically 
associated with introductory high school algebra courses? 

2. Do Intensified Algebra students show improvement in their 
ability to solve non- routine algebraic problems? 

3. How does IA students’ performance vary across six 
different sub-strands of algebra content? 

Algebra as a Gatekeeper Course 

Recent changes in federal and state education policy call 
for a substantial increase in the breadth and depth of 
mathematical knowledge students must acquire in order to 
graduate from high school. A growing number of states now 
require all students to master—on an exit examination—the 
content of Algebra I and Geometry (Kober et al., 2006), and 
these requirements are increasing. Currently, over 40 states and 
the District of Columbia have adopted the Common Core State 
Standards for Mathematics (CCSSM), which call for high 
school graduates to complete three years of rigorous 
mathematics courses, including content typically taught in 
Algebra II, to ensure they are college- and career-ready 
(Council of Chief State School Officers, 2010). 

One reason for these changes is that success in 
mathematics has important consequences for students’ college 
and career options. In fact, research suggests courses, such as 
Algebra I, serve as gatekeepers to more advanced mathematics 
and can affect mathematics achievement in high school and 
beyond (Adelman, 2006; Ma & Wilkins, 2007). In addition, the 
National Educational Longitudinal Study (NELS) indicated 
students taking rigorous high school mathematics courses are 
much more likely to go to college than those who do not 
(Chapman, Laird, Ifill, & KewalRamani, 2011). The NELS 
data specifically showed 83 percent of students that took 
Algebra I and Geometry went to college within two years of 
graduating from high school. This percentage dropped to 36 


5 



Mara V. Martinez, John Bragelman and Timothy Stoelinga 


percent for those who did not take Algebra I and Geometry. 
Identifying and testing the impact on student learning of 
programs intended to support underprepared students’ success 
in algebra is a key to aiding students in attainment of 
subsequent college and career opportunities. 

Students' Difficulties in Learning Algebra 

Over the past two decades, the learning and teaching of 
algebra has increasingly become a central component of the 
mathematics education research agenda (Gutierrez & Boero, 
2006; Stacey, Chick, & Kendal, 2004). As discussed earlier, 
algebra is often considered a gatekeeper to accessing, and 
ultimately understanding, more advanced mathematics 
(National Council of Teachers of Mathematics, 2009; National 
Mathematics Advisory Panel, 2008; U.S. National Research 
Council, 2001). Given the importance of algebra in school 
mathematics, researchers have documented extensively the 
difficulties students have encountered in learning algebra. For 
example, research has shown students do not seem to 
understand the equal sign indicates a relationship between the 
quantities on both sides of the sign, but rather believe it only 
represents a unidirectional operator that produces an output on 
the right side resulting from the input on the left (e.g., Booth, 
1984; Kieran, 1981; Vergnaud, 1985). 

Other research has demonstrated students often focus on 
finding particular answers (e.g., Booth, 1984), do not recognize 
the use of commutative and distributive properties in their work 
on algebra (e.g., Boulton-Lewis et al., 2001; Demana & 
Leitzel, 1988; MacGregor, 1996), and do not use mathematical 
symbols to express relationships among quantities (e.g., 
Vergnaud, 1985; Wagner, 1981). Still, other researchers have 
found students often do not comprehend the use of letters as 
generalized numbers or as variables (e.g., Booth, 1984; 
Kuchemann, 1981; Vergnaud, 1985), have difficulty operating 
on unknowns (e.g., N. Bednarz, 2001; Nadine Bednarz & 
Janvier, 1996; Filloy & Rojano, 1989; Steinberg, Sleeman, & 
Ktorza, 1990) , and often fail to understand that equivalent 
transformations on both sides of an equation do not alter its 
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truth value (e.g., N. Bednarz, 2001; Nadine Bednarz & Janvier, 
1996; Filloy & Rojano, 1989; Steinberg et al., 1990) . 
Together, this body of research illustrates that students have 
difficulty engaging with algebraic concepts typical of Algebra I 
courses. 


IA Design Principles 

In response to the high failure rates encountered in algebra 
nationally, many U.S. school districts adopted the strategy of 
increasing the amount of instructional time in algebra for 
underprepared students (i.e., making Algebra 1 a double-period 
course). Despite the extra time in algebra classes, however, few 
districts implementing the strategy realized anticipated 
improvement in 9th grade algebra performance. Moreover, 
Algebra 1 courses for underprepared students typically consist 
of watered-down algebra content (Ansalone, 2001; Schmidt, 
2008) along with focus on ineffective strategies for promoting 
remedial mathematics skill development, e.g., reteaching 
prerequisites as new learning (Finnan & Swanson, 2000; 
Silver, 1998). The result is, even if students pass these less 
rigorous Algebra 1 courses, they are ill-prepared for success in 
subsequent mathematics courses (ACT, 2007; Adelman, 2006). 
To address these broad challenges, a research and development 
project was initiated in 2008 to increase the success rates of 9th 
grade students who are enrolled in Algebra 1 but were 
significantly underprepared for high school mathematics. The 
project used a design-based research approach (Bannan-Ritland 
& Baek, 2008; Barab & Squire, 2004) to develop and study a 
comprehensive algebra program for students 1-3 years behind 
based on middle school mathematics achievement measures 
and/or diagnostic assessments upon entering high school that 
could be successfully implemented at scale. The resulting 
program, Intensified Algebra I (IA), is a comprehensive 
program for students enrolled in double-period algebra classes, 
used by 38,000 students and 925 teachers in 420 schools in 15 
states as of 2014-15. The program blends text-based and 
technology-based interfaces for students and teachers, and 
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provides teachers with a comprehensive set of professional 
development supports. 

Algebra Core and Functions-based Approach 

To counter the remediation focused, pre-algebraic 
approaches typically enacted in algebra classrooms with 
underprepared learners, the scope and sequence of Intensified 
Algebra was developed to be commensurate with the content of 
a typical, introductory algebra course, and was based on a 
review of state and national standards and recommendations 
(e.g., Achieve, 2008; College Board, 2006; National Council of 
Teachers of Mathematics, 2000; National Mathematics 
Advisory Panel, 2008). Subsequent revisions to the program 
reflect the grade-level content of the Common Core State 
Standards for Mathematics (CCSS-M) (National Governors 
Association Center for Best Practices & Council of Chief State 
School Officers, 2010), and other college readiness standards. 

In addition to inclusion of on-grade-level-content, the 
algebra core in Intensified Algebra also emphasizes explicit 
connections between algebraic procedures and their underlying 
concepts. The value of students’ development of conceptual 
understanding has long been supported in mathematics 
education, with particular benefits noted for struggling students 
in mathematics (Boaler & Humphreys, 2005). Conceptual 
development is facilitated by drawing students’ attention to 
connections among ideas, facts, and procedures (Hiebert & 
Grouws, 2007; Kilpatrick, Swafford, & Findell, 2001) as well 
as their fluency with multiple representations (Brenner et al., 
1997; Koedinger & Nathan, 2004). Based on these principles, 
IA lessons are generally sequenced so students first develop 
foundational concepts by working with contextual 
representations early in a unit, and then progress to more 
abstract, symbolic procedures later in the unit when they can be 
built upon an established conceptual foundation. This 
progression is characteristic of a functions-based approach 
(Yerushalmy & Chazan, 2002), in which ideas about growth 
patterns, relationships, and qualitative behavior of functions are 
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established prior to performing symbolic manipulations with 
those functions. 

Finally, the types of activities presented characterize the 
algebra core of the program. Research on mathematical 
learning has established conceptual understanding develops 
through high cognitive demand tasks that involve important 
mathematics, enacted in a way that maintains their level of 
cognitive demand (Bottge, Heinrichs, Chan, & Serlin, 2001; 
Henningsen & Stein, 1997; Stein, Grover, & Henningsen, 
1996). IA provides instructional tasks and professional support 
emphasizing the enactment of high cognitive demand tasks in 
the classroom, and leveraging the extended time for partner- 
based exploration subsequent connection to formal 
mathematical ideas. 

Architecture of Supports 

Research on mathematics learning among underprepared 
students has highlighted promising areas for providing support 
for their learning of rigorous mathematics. Substantial evidence 
indicates struggling learners characteristically experience 
difficulties with aspects of executive functioning, including 
memory, attention, and self-regulation (Montague & 
Applegate, 1993; Swanson & Jerman, 2006; Swanson & Saez, 
2003). Further, these students typically demonstrate a limited 
repertoire of metacognitive strategies and low motivation 
(Montague, 2007). They also demonstrate difficulty organizing 
and interpreting information and complex processes (Swanson 
& Deshler, 2003; Swanson & Hoskyn, 2001). Metacognitive 
and cognitive strategies support students becoming self- 
directed learners promote students’ problem solving 
capabilities and mathematics achievement (Bransford, Brown, 
& Cocking, 2000; Fuson, Kalchman, & Bransford, 2005). 

In addition, students’ beliefs about their intelligence—as a 
fixed trait (fixed mindset) or one that can grow over time 
(malleable mindset)—influence their motivation to engage in 
academic tasks, and consequently, their success, especially in 
challenging subjects (Dweck, 2007; Good & Dweck, 2006). 
Malleable-mindset interventions, which explicitly teach 
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students about the brain, its functions, and intellectual 
development is the result of effort and learning, have increased 
student achievement in middle school mathematics (Aronson, 
Fried, & Good, 2002; Blackwell, Trzesniewski, & Dweck, 
2007; Good, Aronson, & Inzlicht, 2003). 

Based on these identified areas of need, the IA design 
integrates several foundational support structures. To help 
students organize and self-monitor their learning, the program 
integrates well-defined, daily learning routines, as well as a set 
of graphic organizers that make students’ mathematical 
thinking visible (Swanson & Deshler, 2003; Swanson & 
Hoskyn, 2001). To address students’ misconceptions and 
provide just-in-time opportunities for review and practice 
(Caple, 1996; Rittle-Johnson, Siegler, & Alibali, 2001; Rohrer 
& Taylor, 2007), daily, timely, spaced practice is provided 
specifically addressing skills needed for new learning. Teacher 
supports and worked examples also provide ways to address 
common errors and misconceptions as they arise, rather than 
through the wholesale reteaching of previously learned content. 
To increase students’ abilities to make conceptual connections 
and engage with difficult content, visual animations and 
representations are integrated into daily instruction. These are 
aimed specifically at connecting various representations for the 
foundation of the functions-based approach. 

Finally, a set of social-motivational tools have been 
integrated into the program, based on research about students’ 
beliefs about themselves, and its role in academic learning and 
motivation (Aronson, Fried, & Good, 2002; Blackwell, 
Trzesniewski, & Dweck, 2007; Dweck, 2007; Fuson, 
Kalchman, & Bransford, 2005; Good & Dweck, 2006. Building 
from this research, the Dana Center and Agile Mind developed 
the Academic Youth Development (AYD) program, for 
entering ninth graders to support students’ aspirations for high 
achievement by teaching them theories of malleable 
intelligence, strategies for goal setting and effective effort, how 
to learn, and effective communication strategies and applying 
them to mathematics. Collectively, these AYD materials have 
been adapted and strategically incorporated into IA’s scope and 
sequence to directly address issues of motivation and mindset 
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associated with learning disabilities students and underprepared 
learners. 


Methods 


Participants 

A total of 153 students of diverse background from 28 
urban schools across the United States took part in the IA 
course during the 2011-2012 academic year. Majority of the 
schools were located in Illinois and Texas and most of the 
students were in the ninth-grade. Parents’ consent and students’ 
assent rates were low in participating schools, which, in turn, 
resulted in a relatively small sample. Low rates were due 
mostly because of non-retumed parental consent forms despite 
multiple attempts by the research team. For analysis purposes, 
the sample size is reduced to 78 students who were present 
during the multiple administrations of each of the two 
assessments. 

Instruments 

Multiple-choice assessment. The multiple-choice 
assessment was designed to measure students’ proficiency with 
concepts and skills of the Algebra I course. Specifically, it was 
modeled after the Acuity™ Algebra Proficiency Test 1 because 
Acuity™ is a widely used comprehensive instrument for 
Algebra I concepts and skills. The Acuity™ Algebra 
Proficiency Test was developed and published by 
CTB/McGraw-Hill in 2007 and consists of 32-items intended 
for students who have completed Algebra I. 

To construct two parallel forms of the IA multiple-choice 
assessment, we first surveyed all released National Assessment 
of Educational Progress (NAEP) items matching the math 


'AgileAssessment is a bank of formative assessment items developed by our 
IA technology partner, Agile Mind, that are related to their other middle and 
high school mathematics courses. 
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concept and/or skill represented in a particular Acuity™ item. 
If no comparable item was found in this item bank we selected 
items from the AgileMind Assessments item bank. Two 
researchers constructed the tests. For each Acuity™ item, one 
researcher proposed an initial selection after which the other 
researcher accepted or rejected the selection. If the item was 
rejected an alternative was suggested. This iterative process 
continued until both researchers agreed on each item for both 
forms/versions of the test. Two sample problems are shown in 
Figure 1. 


Sample Problem 1 

A plumber charges customers $48 for each hour worked, plus an additional 
$9 for travel. If h represents the number of hours worked, which of the 
expressions could be used to calculate the plumber’s total charge in dollars? 
(Students select the correct answer from five different choices) 


Sample Problem 2 


X 

y 

0 

-l 

l 

2 

2 

5 

3 

8 

10 

29 


Which of the following represents the relationship between x and y shown in 
the table above? (Students select the correct answer from five different 
choices) 

Figure 1. Sample Items in the IA Multiple Choice Assessment. 

In addition to measuring change in students’ overall 
performance according to the multiple choice assessment, we 
also wanted to investigate the extent to which there were 
specific areas within the content of Algebra I that showed 
greater change from pre- to post- assessment. To do so we 
grouped items according to the following content areas: 
Variables, Expressions, Formulas (VEF), Solving Linear 
Equations (SLE), Graphing Linear Equations (GLE), Functions 
and Graphs (FG), Quadratic Equations and Functions (QEF), 
and Geometry (G). 
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Constructed response assessment. The constructed 
response assessment was designed to provide a measure of 
students’ understanding of a variety of concepts and skills 
central to Algebra I. To design the constructed response IA 
assessment, we first surveyed tasks from the MARS project 
(http://map.mathshell.org/materials/index.php) through the 
Silicon Valley Mathematics Initiative 

(http://www.svmimac.org/). Two researchers constructed the 
two parallel forms of the assessment. For each form, one 
researcher proposed an initial selection after which the other 
researcher accepted or rejected the selection. If a task was 
rejected an alternative was suggested. This iterative process 
continued until both researchers agreed on each task for both 
forms of the assessment. 

Table 1. 


Selected Tasks and Their Corresponding Skills 


Form 

Task Name 

Skills and Concepts 

C 

Toy Trains 

Find and use a number pattern 

Find an algebraic expression for a number pattern 


Lawn 

Mowing 

Solve a practical problem involving ratios 

Use proportional reasoning 


Going to 

Town 

Interpret and complete a distance/time graph for a 
described situation 


Vacations 

Analyze relationships using graphs and algebra 


Picking 

Apples 

Work out costs from given rules 

D 

Patchwork 

Quilt 

Recognize and extend a number pattern 

Express a rule using algebra 


Photographs 

Use proportion in a real life geometric context 


Bike Ride 

Interpret a distance/time graph 


Shelves 

Solve problems in a spatial context 

Identify and distinguish the four point graphs 
related to this situation 


Buying Chips 
and Candy 

Form and solve a pair of linear equations in a 
practical situation 
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Selected tasks and skills were intended to be measured are 
included in Table 1. Specifically, two tasks (i.e., Toy Trains 
and Patchwork Quilt), one in each form of the assessment, 
were intended to measure students’ skills and knowledge about 
linear functions. Each of the constructed response problems has 
several questions or prompts organized in sub-parts. 

Data Collection 

The two versions of the multiple-choice assessment and of 
the constructed response assessment (i.e., Form C and Form D) 
were administered at the beginning (pretest) and end (posttest) 
of the 2011-2012 school year. There were 33 multiple-choice 
items on each version of the multiple-choice test. 2 There were 
five constructed response problems on each version of the 
constructed response assessment, each consisting of multiple 
parts. Students were randomly given a version of the pretest, 
and then given the alternate version for the posttest. For 
purposes of this study, we analyzed data from students who 
took Form C for the pretest and Form D for the posttest and 
vice versa excluding students who took the same form as both 
pre and posttests. This reduced the total number of students for 
analysis purposes to 153 on the multiple-choice assessment of 
which 87 took Form C for the pretest and Form D for the 
posttest and 66 took Form D for the pretest and Form C for the 
posttest. On the constructed response assessment, the same 
procedure reduced the total sample to 128, 72 who went from 
Form C to Form D, pre to post, and 56 students who went from 
Form D to Form C, pre to posttest. 

The pretest administration occurred at the beginning of the 
fall semester, before much instruction on the algebra content 
had occurred. The posttest was administered at the end of the 
school year when most of the instruction was complete. 


2 The original test had 32 items, after reviewing pilot data, one item was 
given two question prompts. 
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Reliability of Instruments 

Reliability of the multiple choice and constructed response 
assessments was determined using Cronbach’s alpha to 
measure internal consistency. Table 2 shows the Cronbach’s 
alpha for the assessments varied from .48 to .80 with lower 
reliability at pretest than posttest. This is expected given the 
small sample as well unfamiliarity with some of the specific 
content prior to actual instruction. However, the reliabilities at 
posttest for both forms are acceptable and in the range reported 
for the Acuity instrument following instruction in Algebra I. 

Table 2. 


Reliability Analysis of Multiple Choice Assessment 


Assessment 

No. of 
Items 

Mean 

N 

Std. Deviation 

Cronbach’s 

a 

Pretest Form C 

33 

0.3176 

87 

0.1139 

0.48 

Pretest Form D 

33 

0.3512 

66 

0.1345 

0.62 

Posttest Form C 

33 

0.4600 

66 

0.1764 

0.8 

Posttest Form D 

33 

0.4482 

87 

0.1579 

0.72 

Table 3 

shows the Cronbach’s alpha for the constructed 

response assessments 

varied 

from 

.83 to .90, 

suggesting 


students were consistent in their performance on Form C and 
Form D on both the pretest and posttest. 

Table 3. 


Reliability Analysis of Constructed Response Assessment 


Assessment 

No. of 
Items 

Mean 

N 

Std. Deviation 

Cronbach’s 

a 

Pretest Form C 

44 

0.3730 

72 

0.1732 

0.87 

Pretest Form D 

37 

0.3797 

56 

0.1777 

0.86 

Posttest Form C 

44 

0.4944 

56 

0.2005 

0.90 

Posttest Form D 

37 

0.5351 

72 

0.1849 

0.83 
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Results 

Overall Student Performance 

As mentioned before, due to the difference in sample sizes 
between the group of students who took the multiple choice 
assessment at both pretest and posttest (i.e., 153) and the group 
who took the constructed response assessment at both testing 
times (i.e., 128), as well as variation in the overlap between 
these groups, all further analyses are performed using the 
intersection of the two samples (all students who took both the 
multiple choice and the constructed response assessment at 
both the pretest and the posttest). This reduces the total sample 
size to 78 for purposes of analysis. 

Multiple Choice Assessment 

A repeated measures ANOVA was used to assess student 
performance from pretest to posttest and to assess differences 
between assessment versions and content areas. Results from 
the ANOVA were used to compute effect size, both classic eta 
squared (r|2) and partial eta squared (qp2). To briefly 
summarize, classic eta squared is a proportion of the total 
variation explained by the factor, and partial eta squared is a 
proportion of the explained variance, excluding variances 
produced by other factors outside of the analysis. Following the 
recommendations of Pierce, Block, and Aguinis (2004) and 
Levine and Hullett (2002), both are reported as each follows a 
different index when determining the strength of association. 
Forty-seven students took version C for the pretest and version 
D for the posttest, while 31 students took version D for the 
pretest and version C for the posttest. There was not a 
statistically significant main effect for version (F[l,76] = 
1.591, p > .05). In addition, there were no statistically 
significant interactions found between time and version 
(F[l,76] = .252, p > .05), between content area and version 
(F[5,72] = 2.210, p > .05), and between time, content area, and 
version (F[5,72] = 1.984, p > .05). This outcome suggests no 
difference in student performance exists between versions for 
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the pretest or posttest, overall or by content area. As such, 
version has been removed from the subsequent analyses. 

Overall student performance at pretest was found to be 
significantly different from posttest (F[l,77] = 31.780, p < 
.001, r|2 = 0.0553, rip2 = 0.292), with students performing 
significantly better on the posttest. As seen in Table 4, students 
averaged 31.15% correct on the pretest with a standard 
deviation of 11.08%, and 44.56% correct on the posttest with a 
standard deviation of 16.82%, showing an average gain of 
13.41% over the intervention period. Effect sizes of pretest to 
posttest were very large, accounting for 5.53% of the total 
variation in student performance and 29.2% of the variation 
excluding factors not accounted for in the analysis. Of the 77 
students in the sample, 70.13% improved from pretest to 
posttest on the multiple-choice assessment. 

Table 4 


Relative Improvement Results 


Test 

Mean 

Standard Deviation 

Pretest 

0.3115 

0.1108 

Posttest 

0.4456 

0.1682 


Figures 2 and 3 show the distribution of the students’ 
relative scores on the pretest and posttest. 
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Figure 2. Distribution of Pretest Scores 
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Figure 3. Distribution of Posttest Scores 
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As mentioned previously, the multiple choice items were 
categorized by content area, resulting in six domains: four 
items as Variables, Expressions, and Functions (VEF); eleven 
items as Solving Linear Equations (SLE); four items as 
Graphing Linear Equations (GLE); eight items as Functions 
and Graphs (FG); four items as Quadratic Equations and 
Functions (QEF); and two items as Geometry (G). Differences 
in student performance among these categories were found to 
be statistically significant (F[3.841,295.758] = 13.696, p < 
.001, r|2 = 0.0669, r\p2 = 0.151). For both this main effect and 
the interaction between category and test, Maulchy’s Test of 
Sphericity was violated (X2= 57.878, df = 14, p < .001). As 
such, results from the Greenhouse-Geisser correction were 
used. Students’ difference in performance between content 
areas had a medium effect size (r|2 = 0.0669), and the results 
suggest content area accounts for 15.1% of the variance in 
students’ performance excluding other factors. Table 5 shows 
mean performance by content area for the pretest and posttest. 
Students performed highest on Solving Linear Equations, 
Graphing Linear Equations, and Functions & Graphs, 
averaging 50.3%, 46.2%, and 51.9% respectively. 


Table 5 

Descriptive statistics by content area and testing period 




Pie 




Post 



Mean 

S.D. 

Mean 

S.D. 

N 

F[ 1,77] 

Sig. 

VEF 

0.324 

0.239 

0.343 

0.288 

78 

.231 

.632 

SLE 

0.373 

0.163 

0.503 

0.215 

78 

18.179 

.0001 

GLE 

0.269 

0.227 

0.462 

0.261 

78 

24.706 

.0001 

FG 

0.298 

0.188 

0.519 

0.258 

78 

47.978 

.0001 

QEF 

0.250 

0.221 

0.304 

0.244 

78 

2.109 

.151 

G 

0.250 

0.288 

0.301 

0.283 

78 

1.398 

.241 


The interaction between content area and performance at 
pretest and posttest was significant (F[4.335,333.820] = 6.725, 
p < .001, r|2 = 0.0264, rip2 = 0.069—results from Greenhouse- 
Geisser correction reported) indicating that student 
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performance changes from pretest to posttest differed 
depending on the content area. The interaction between content 
area and change from pre- to post- had a small effect size (r)p2 
= 0.0264), and the results suggest content areas account for 
6.9% of the variance in students’ performance excluding other 
factors. Figure 2 shows students’ performance changes from 
pretest to posttest by content area. Three categories show a 
significant improvement from pre to post: 

GLE (F[ 1,77] = 18.179, p < .001, i\2 = 0.049, qp2 = 0.243 ), 
FG (F[l,77] = 24.706, p < .001, p2 = 0.065, pp2 = 0.384), and 
SLE (F[l,77] = 47.978, p < .001 , i\2 = 0.023, rip2 = 0.191). 

Student performance on Graphing Linear Equations had a 
medium effect size (r\2 = 0.049), and the domain accounts for 
24.3% of the variance in students’ performance excluding other 
factors. Student performance on Functions and Graphs had a 
mediun effect size (r\2 = 0.065), and the domain accounts for 
38.4% of the variance in students’ performance excluding other 
factors. Last, student performance on Solving Linear Equations 
had a small effect size (r\2 = 0.023), and the domain accounts 
for 19.1% of the variance in students’ performance excluding 
other factors. Two categories show a small improvement from 
pre- to post- that was not significant: QEF and G. And finally, 
one category shows almost no change: VEF. Thus, the IA 
program had the most impact on the Graphing Linear 
Equations, Functions and Graphs, and Solving Linear 
Equations content areas. These are central to any curriculum in 
Algebra I, in general, and IA in particular. We expected to find 
a significant change in performance for the Variables, 
Equations and Formulas content area but failed to do observe 
such an outcome. 
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- VEF - SLE -GLE - FG - QEF - G 

Figure 4. Performance by Category and Testing Period 

Constructed Response Assessment 

A repeated measures ANOVA was used to assess student 
performance at pretest and posttest and to assess differences 
between instrument versions on the constructed response 
assessment. No significant main effect was found for version 
(F[l,76] = .190, p > .05). Further, no significant interaction was 
found between time and version (F[l,76] = .102, p > .05). This 
outcome suggests no difference exists between student 
performance on the pretest or posttest between versions. As 
such, version has been removed from the subsequent analyses. 

Overall the change in student performance from pretest to 
posttest was found to be significant (F[l,77] = 43.346, p < 
.001, rjp2 = 0.360) with students performing significantly better 
on the posttest. The change over time from pre- to post- had a 
very large effect size (t]p2 = 0.360). Of note, only partial eta 
squared is reported, as the analysis included one within-subject 
effect, making eta squared and partial eta squared identical. As 
seen in Table 6, students averaged 34.48% correct on the 
pretest with a standard deviation of 17.50% and 50.85% on the 
posttest with a standard deviation of 18.62%, showing a 
16.37% improvement over the intervention period. Of the 77 
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students in the sample, 84.42% improved from pretest to 
posttest on the constructed-response assessment. 

Table 6. 


Constructed Response Performance 


Test 

Mean 

Standard Deviation 

Pretest 

0.3448 

0.1750 

Posttest 

0.5085 

0.1862 


Due to the nature of the constructed response assessment 
most of the tasks were too complex conceptually to categorize 
by a single content area. Furthermore, only one task on each 
version assessed Solving Linear Equations (SLE): The Toy 
Trains task on version C and the Patchwork Quilt task on 
version D. A repeated measures ANOVA was performed on 
this subset of tasks to examine differences between the two 
versions as well as growth over the intervention. Results 
indicated that while the students’ improvement from pretest to 
posttest on the constructed response task on Solving Linear 
Equations was significant (F[l,76] = 11.061, p = .001), the 
interaction between performance pre to posttest and the version 
was also significant (F[l, 76] = 26.449, p = .000). One version 
was easier for the students than the other; in particular, students 
performed significantly better on the Toy Trains task on 
version C than on the Patchwork Quilt task on version D, 
whether on the pretest or posttest. As such, performance results 
at the task level for assessing content area change are not 
meaningful. 


Implications 

This study provides supporting evidence regarding the 
potential positive impact of adopting a comprehensive 
curriculum program for double-period algebra in its early 
stages of implementation. The program is specifically designed 
to support underprepared students’ learning of rigorous 
algebra. Students who are enrolled in double-period algebra 
typically have experienced a persistent lack of success in 
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school mathematics in their years leading up to 9th grade, and 
as a result may often see mathematics—particularly 
introductory algebra—as an endeavor neither worthwhile nor 
holding the possibility of success. With the implementation of 
double-period algebra policies, teachers face the daunting 
challenge of helping students regain ground in mathematics, 
while managing longer class periods with students who may no 
longer be invested in the enterprise. 

While results from this study are from a relatively early 
iteration of IA’s development/implementation cycle and are 
generated from a relatively small pilot program, the results 
demonstrate that students who have experienced persistent 
difficulty in learning mathematics are able to make significant 
gains with on-grade-level content in IA classrooms. These 
gains are evident in their performance both on multiple-choice 
items emphasizing discreet algebraic skills and procedures, and 
on constructed-response items emphasizing solving of open- 
ended, non-routine algebraic problems. As an ongoing design- 
based research project, IA continues through iterative cycles of 
implementation, revision, and testing. The results of this study 
therefore provide a baseline to compare against measures of 
student learning from future IA implementations that 
incorporate ongoing adjustments to the program, varied 
contexts of implementation, and potentially larger 
implementation samples. 

Beyond the overall assessment results, the differences in 
student performance across specific sub-strands of content 
suggest students in IA classrooms may be learning more 
content in some domains compared with others. In particular, 
the post-hoc analyses of content-area performance on the 
multiple-choice instruments show significant pre to posttest 
gains in mean scores for the domains of Solving Linear 
Equations, Graphing Linear Equations, and Functions and 
Graphs in the multiple-choice assessment. Notably, extensive 
opportunities to learn in these three domains are provided 
throughout the scope and sequence of the IA program, 
indicating students’ performance in these areas may benefit 
from this programmatic emphasis. Conversely and not 
unexpectedly, statistically significant gains in mean scores 
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were not achieved in the Geometry domain, where far fewer 
opportunities to learn are provided, based on the algebraic 
focus of the program. 

Further consideration, however, needs to be given to the 
algebra-related domains of Variables, Equations, and Functions 
(VEF) and Quadratic Equations and Functions (QEF). In both 
of these domains, students did not demonstrate significant 
changes from pre to posttest in the post-hoc analyses as 
expected. One possible explanation to be explored involves the 
positioning and treatment of these sub-strands within the IA 
scope and sequence. The VEF domain, for example, is 
emphasized early in the program, but it is unclear the degree to 
which opportunities to re-visit content in this domain are 
offered to students throughout the curriculum. In a similar vein, 
the QEF domain is addressed in the latter units of the program, 
where pacing issues often preclude implementation before the 
end of the academic year, particularly in the early years of 
program adoption. A targeted exploration of how instruction is 
implemented across particular content areas in IA, which 
focuses on both descriptive and quantitative accounts of 
opportunities to learn for students, may provide further 
understanding of differences of learning across algebraic 
content domains. 

This direction of research could potentially be extended to 
contribute to the field more generally as well. By 
systematically tracing students’ learning throughout the IA 
course to empirically validate or modify underlying learning 
trajectories for a functions approach to Algebra I, ongoing 
research of IA implementations could contribute to the 
development of more effective resources and tools for both 
instruction and assessment, particularly in the algebraic 
domains. A report from the Center for Policy and Research in 
Education (Daro, Mosher & Corcoran, 2011) highlighted the 
lack of learning trajectories for high school mathematical 
concepts, and has called for studies constructing and 
empirically validating such learning trajectories. Such research 
could contribute to filling this empirical gap. 

In the broader research context, the learning outcomes 
demonstrated in this study add to a growing understanding in 


24 



Underprepared Students’ Performance on Algebra 


the field about the algebraic learning of underprepared students 
in double-period contexts. As implementations of double¬ 
period policies continue to be adopted by school districts, IA 
and other programs designed for students participating in 
extended models of instruction (e.g., Dubinsky & Wilson, 
2013) will continue to provide a test-bed for studying the 
teaching and learning of rigorous algebra for underprepared 
learners in these contexts. Furthermore, while this study 
provides evidence underprepared students benefit from 
participating in a single year of double-period Algebra I 
instruction with IA, it does not provide a comparison to any 
other double-period algebra interventions currently being 
implemented in schools. In particular, further research is 
needed to address how performance of underprepared students 
in IA compares to similar students in classrooms where other 
programs/models are being implemented. 3 Beyond 
comparisons of standardized performance measures, any such 
exploration should also include a more nuanced investigation 
into what particular mathematics students learn in the context 
of each intervention, how students and teachers engage with 
the opportunities to learn provided, what affordances and 
benefits to students and teachers each approach provides, and 
under what contexts and conditions. 

Finally, research on how to support underprepared ninth 
graders’ success in Algebra I has increased importance as states 
and districts implement the Common Core State Standards for 
Mathematics (CCSS-M) (CSSO, 2010). The CCSS-M make 
major advances in K-12 mathematics curricula in a number of 
ways, one of which is including the study of linear functions 
and equations as core content of Grade 8 mathematics rather 
than as content for Algebra I. The study of linear functions has 


3 E.g., See Transition to Advanced Mathematics (Neild, Byrnes & Sweet, 

2011) , a program developed at Johns Hopkins University, organized with a 
first semester of pre-algebra followed by a second semester of algebra; or the 
Talent Development Program developed at Johns Hopkins University; or the 
Transition to Algebra program, modeled as an additional support course to 
accompany a standard Algebra I course (Education Development Center, 

2012 ) . 
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historically been addressed as part of the introductory, 9th- 
grade algebra curriculum, with typically only a subset of 
students encountering the topic in an 8th-grade algebra course. 
The CCSS-M has shifted much of the introductory algebra 
content, including linear functions, into the standards for Grade 
8. This shift is likely to result in more students being 
underprepared to learn linear functions (i.e., students who were 
underprepared to learn algebra in 9th grade will be more so in 
8th grade, and likely in increasing numbers). Thus, learning 
how to better support underprepared students to understand 
linear functions and equations is a critical issue for CCSS-M 
implementation, as well as for mathematics education in 
general. 
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